CSC 453 - day12q

Operating Systems

January 1, 2026

This class

Who am I?

  • https://pschmitt.net
  • Academics
    • PhD at UCSBsss
    • Researcher (summers) at ICSI
    • Postdoc then research faculty at Princeton
    • Some other stops… USC, Stanford, UH
  • Privacy startup co-founder & CEO, measurement startup COO
  • Main focuses
    • Integrating privacy and security into systems that we all use
    • ML for network management
    • Connectivity / privacy in challenging environments

This class

Who are you?

  • Name / Major / Year
  • Where do you identify as your hometown?

How to succeed in this class

  • Turn things in on time
  • Attend class
    • … on time. Lectures will start at 10 after the hour
  • Ask questions
    • I’m very flexible about how much we cover this semester
    • I would rather teach less and have everyone understand it
    • Our back-and-forth during class is the one of the few indicators I have of how much you are absorbing
  • Talk to me if you are struggling

What are we doing here?

  • How many of you have participated in OS development?
  • How many of you regularly program in languages that use operating system abstractions directly?
  • And C is a decreasingly popular language!
  • So why study operating systems? Why is this class even offered? Why is it required?

Why take this class

  • You are required to do so in order to graduate
  • Reality: this is how computers really work, and as a computer scientist or engineer you should know how computers really work
  • Ubiquity: operating systems are everywhere and you are likely to eventually encounter them or their limitations
  • Beauty: operating systems are examples of mature solutions to difficult design and engineering problems. Studying them will improve your ability to design and implement abstractions

Course progression

  • Introduction to operating system abstractions and structures
  • Abstracting and multiplexing:
    • CPU: interrupts, context, threads, processes, processor scheduling, thread synchronization, deadlocks
    • Memory: memory layout, address translation, paging and segmentation, address spaces, translation caching, page fault handling, page eviction, swapping
    • File systems and storage: disk scheduling, on-disk layout, files, buffer cache, crash and recovery
    • Security (time permitting)

Background

Intro

  • What is an operating system?
  • What do they do?
  • How did OSes evolve?
  • How do OSes manage access to resources?

What is an operating system?

  • Operating System:
    • A computer program that
    • multiplexes hardware resources and
    • implements useful abstractions.
  • The OS is just another computer program. Has the highest privilege. The core of the OS consists of the kernel
  • Multiplexing allows multiple people or programs to use the same set of hardware resources - processors, memory, disks, network connection - safely and efficiently.
  • Abstractions simplify the usage of hardware resources by organizing information or implementing new capabilities.

OS history - how did we get here?

  • Started out as libraries to provide common functionality across programs
    • See any issues with this approach?
  • Later, evolved from procedure call to system call
    • What’s the difference?
  • Evolved from running a single program to multiple processes concurrently
    • New issues to solve?

Types of operating systems

  • Desktop
  • Time share/Mainframe
  • Mobile
  • Web
  • Real-time
  • Embedded
  • Virtual Machines
  • Q: How do the requirements differ?

Tl;dr

OSes continue to evolve as environments (i.e., constraints) change

What isn’t clear?

Comments? Thoughts?

OS abstractions

Abstractions

  • Abstractions simplify application design by:
    • hiding undesirable properties,
    • adding new capabilities, and
    • organizing information
  • Abstractions provide an interface to application programmers that separates policy—what the interface commits to accomplishing—from mechanism—how the interface is implemented.

What are the abstractions?

  • CPUs
    • Processes, threads
  • Memory
    • Address space
  • Disk
    • Files

Example OS abstraction: file systems

  • What undesirable properties do file systems hide?
    • Disks are slow!
    • Chunks of storage are actually distributed all over the disk
    • Disk storage may fail!
  • What new capabilities do files add?
    • Growth and shrinking
    • Organization into directories, searchability
  • What information do files help organize?
    • Ownership and permissions
    • Access time, modification time, type, etc.

Abstraction tradeoffs - discussion

  • Identify undesirable properties hidden by, new capabilities added, and info organization provided with these abstractions:
    • Process / threads
    • Address space

Abstraction pros / cons

  • Advantages of OS providing abstractions?
    • Allow applications to reuse common facilities
    • Make different devices look the same
    • Provide higher-level or more useful functionality
  • Challenges?
    • What are the correct abstractions?
    • How much should be exposed?

OS design requirements - what do we need?

  • Convenience, abstraction of hardware resources for user programs
  • Efficiency of usage of CPU, memory, etc.
  • Isolation between multiple processes
  • Reliability, the OS must not fail
  • Other:
    • Security
    • Mobility

What isn’t clear?

Comments? Thoughts?

Resource management

OS as a resource manager

  • Another view: resource manager - shares resources “well”
  • Advantages of the OS managing resources?
    • Protect applications from one another
    • Provide efficient access to resources (cost, time, energy)
    • Provide fair access to resources
  • Challenges?
    • What are the correct mechanisms?
    • What are the correct policies?

Resources are managed via services

  • Program Execution (loading, running, monitoring, terminating)
  • Performance (optimizing resources under constraints)
  • Correctness (overseeing critical operations, preventing interference)
  • Fairness (access to and allocation of resources)
  • Error detection & recovery (network partition & media failure)

Services (cont’d)

  • Communication (inter-process, software-to-hardware, hardware-to-hardware, system-to-system, wide-area)
  • I/O: reading & writing, support for various mediums, devices, performance, and protections
  • Data Organization (naming), Services (search) & Protection (access control)
  • Security (isolation, enforcement, services, authentication, accounting and logging, trust)
  • User interfaces (command-line, GUIs, multiple users)

Each service has challenges and tensions

Example 1: We have limited RAM, and we want to run more programs that can be stored.

  • How do we allocate space?
  • Who stays?
  • Who goes?
  • What if we’re wrong?
  • What if the system is under extremely heavy load?
  • Is there a way to predict the future?

Each service has challenges and tensions

Example 2: We have two process (producer / consumer); how do they communicate?

  • Message passing? Shared memory?
  • How do they synchronize?
    • How to we prevent over-production? Over-consumption?
    • Context-switching?

What isn’t clear?

Comments? Thoughts?

Kernel basics

Briefly: what happens at boot?

  • BIOS / UEFI initializes devices and finds the next program to run
  • Bootloader: simple code to initialize the system, load the kernel
    • Why do we need a bootloader?
      • Safe mode
  • Kernel loads
    • First process: init (example: systemd), pid == 1
      • Parent of all processes
      • Init starts system daemons (services provided outside of the kernel)

Paradigms

  • The OS offers a number of services. What should go in the kernel?
    • IPC
    • VFS
    • File system
    • Scheduler
    • Virtual Memory
    • Device drivers

Monolithic kernels

  • Oldest, very common design (Linux, Windows 9x, BSDs)
  • Single piece of code in memory
  • Limited information hiding
    • One part of the kernel can directly access data and functions of other parts

Monolithic kernels (cont’d)

Q: What happens when you need something new supported (new hardware device, new system call, new filesystem)?

  • Modules (loadable kernel modules - LKMs) allow for flexibility, customization, support
    • Pros?
      • Memory savings
      • Flexibility
      • Minimal downtime
    • Cons?

Layered kernels

  • Dijkstra created the THE OS in the 60s, introducing the concept Each inner layer is more privileged; required a trap to move down layers
  • Hardware-enforcement possible
  • Intel announced in May 2024 that the new architectures will only have rings 0 and 3

Microkernels

  • Popular research area long ago, didn’t win (although people remain interested in the principles)
  • All non-essential components removed from the kernel, for modularization, extensibility, isolation, security, and ease of management
  • A collection of OS services running in user space
  • Downsides?
    • Heavy communication costs through message passing (marshaled through the kernel)

Monolithic vs Microkernel

What isn’t clear?

Comments? Thoughts?

Process basics

Process isolation

Q: How can we ensure that a user process doesn’t harm others?

Dual-mode operation

  • Dual-mode operation allows OS to protect itself and components
    • User mode and kernel mode
  • Mode bit provided by hardware
    • Provides ability to distinguish when system is running user code or kernel code.
    • When a user is running → mode bit is “user”
    • When kernel code is executing → mode bit is “kernel”
  • System call changes mode to kernel, return from call resets it to user
  • Some instructions are only executable in kernel mode

System calls

  • The OS offers a number of services. How do we (applications) interface with them?
    • We don’t want to deal with the details, just the abstraction
    • The OS has ultimate control over these operations
  • System calls are the “language” of communication with the OS
  • Standards
    • Win32 (MS)
    • POSIX (nearly all Unix-based systems)
    • Java API for the JVM

System calls (cont’d)

  • Like a function call, we push arguments onto the stack, then we call into the library that provides the system call
  • Each system call has a special number, placed into a register
  • Executes a TRAP instruction (switch to kernel mode)
  • A logical separation of memory space
  • Kernel’s system call handler is invoked, once done (but may block) may be returned to the process

System calls (cont’d)

What isn’t clear?

Comments? Thoughts?

Assignment 1: SLOsh

SLOsh

  • This assignment should be a refresher of 357 topics
  • Your shell will support:
    • basic command execution
    • built-in commands
    • signal handling
    • I/O redirection
    • piping

Requirements

  • C99 POSIX
  • Must compile / run on the unix* servers
  • You must create a functional Makefile
  • Submit a tar (.tgz) file in Canvas

Command execution

  • Execute external commands
    • Use fork(), exec(), and waitpid() to manage process execution
  • Support command-line arguments
  • Handle both relative and absolute paths to executables
  • Report command execution errors appropriately

Redirection / piping

  • Use open(), dup2(), and close() for file redirection
  • Use pipe() for creating pipes between processes

Signal handling

  • Signals are async. They can happen at any time
  • Certain functions are async-safe, others are not
    • printf() contains internal buffers and state; an interruption can lead to corrupted buffers

Handling SIGINT

  • write() a newline or whatever to signify the signal was caught.
  • If there is not a child running, write() a new prompt
  • Why not just use display_prompt()?
    • If it’s using write() it could be fine. If it’s using printf() it isn’t

Parent vs child

  • The child should immediately reset signal handling to the default

  • Look up signal()’s man

    man 2 signal

Sigaction

  • sigaction() is newer than signal()
  • Offers more control and is more consistent across systems
  • Allows specifying flags
    • SA_RESTART (man 7 signal)
      • Without it, syscalls return with EINTR error if they are interrupted by a signal
      • With it, most syscalls automatically restart
  • Is the POSIX-compliant approach
  • EINTR can still happen, it’s wise to check for it and handle it
    • When reading input from user